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PREFACE 



The Center for Vocational I>c!ur^'ition is indebted to Dr. Rupert Evans for his lecture entitled, 
**VocationaJ Education R&D in tht Past Decade: Implications for the Future." 

Dr. Evans, professor of Vocational and Technical Education, College of Education, University of 
Illinois, discussed the needs for research and development during the next decade. The lecture was 
based on the re.^ults of a study by the Committee on Vocational Education Research and Development 
concl'icted by tlie National Academy of Science. 

Dr. Evans has a diverse background and experience in vocational education, including a high 
school vocational teacher, foreman for General Motors and dean of the College of Education, Uni- 
versity of Illinois, Urbana. 

Completing his Bachelor's degree at Indiana State Teachers College, Dr. Evans received his M.S. 
and Ph.D. from Purdue in 1950. 

In addition to his recently serving as chairman. Committee on Vocational Education Research 
and Development, National Academy of Sciences, Dr. Evans is also chairman of the Illinois Advisory 
Council on Vocational Education anri member of the Illinois Governor's Manpower Advisory Council. 
Dr. Evans also holds membership in a number of professional associations and honoraries. 

Dr. Evans has published numerous articles in the American Vocational Journal, School Shop, 
Review of Educational Research, Nations Schools, and other professional education journals. 

Hi'. rc: ?nt books include: 

CaiU f ^idiication, A Handbook for Implementation and Career Education, What It Is and How 
to Do ft. Olympus, Second Edition, 1974. 

'•Innovations in Vocational Education Which Are Related To Manpower Training," in Manpower 
Policy: Perspectives and Prospects, Seymour Wolfbein, Editor. Temple University Press, 
1973. 

Ca*'eer Education for fhe Middle/Junior High School, Olympus Publishing Company, 1973. 

Vocational Education: Staff Development Priorities for the 70's, National Advisory Council 
on Education Professions Development. U.S. Government Printing Office, 1973. 

Changing the Role of Vocational Teacher Education. McKnight and McKnight, 1971. 

Foundations of Vocational Education, March 1971. Charles Merrill. 

On behalf of The "^^enter and The Ohio State University, I take pleasure in presenting Dr. Evans 
lecture "Vocational rkli ation R&D in the Past Decade: Implications for the Future." 
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VOCATIONAL EDUCATION R & D IN THE PAST DECADE: 
IMPLICATIONS FOR THE FUTURE 



I certainly appreciate this opportunity to talk with you, and I'll enjoy even more responding to 
ycur questions. I don't enjoy lecturing, as such, but I really do enjoy interacting with people, espe- 
cially with knowledgeable people such as the group assembled here. This is almost a return home for 
me; I feel a great kinship towards this place. I chose to spend my most recent sabbatical here and 
thoroughly enjoyed it, so coming back here to talk to you is a particular honor. Serving as chairman 
of the Committee on Vocational Education Research and Development of the National Academy of 
Sciences, however, was a somewhat more dubious honor. I must admit that when I said "Yes" to 
their kind invitation, I didn't realize all that was going to be involved. 

The National Academy of Sciences is a very old, very conservative, and a very prestigious group. 
Just to give you some notion of their conservatism; though they're almost one hundred and fifty 
years old, they have still not gotten around to deciding that the metric system is a good idea. But 
one thing that they are particularly good at is selecting committees. The task that we've faced simply 
could not have been accomplished without the kind of committee membership that we had: Bob 
Taylor was on the Committee, Gordon Swanson was on the Committee, Gloria Cooper was on the 
Committee. Gloria was a Research Coordinating Unit head at the time she was selected, but she later 
moved to the Ohio State Center, so it turned out that we had two representatives on the Committee 
from The Center, but it wasn't done deliberately that way. They and the other people on the Com- 
mittee made the completion of our job possible. 

In his introduction. Bob Taylor mentioned that we were charged with evaluating the last ten 
years of work in vocational education R & D and with making recommendations for the next ten 
years. As nearly as I can tell, the study began with a direct suggestion from Gordon Swanson to the 
U.S. Office of Education. He pointed out that the National Academy was doing a study of manpower 
research, and asked, "Why don't you do something like this for vocational education research?" As 
with many good ideas, you wonder why somebody didn't do it a long time ago. However, it may be 
that after you read our report, you'il wonder why somebody else doesn't do it. Certainly it turned 
out to be a horrendous job. 

This was not the fault of USOE personnel. They certainly were extremely helpful to us; they 
made no attempt to dictate the design of the study or to influence the recommendations or con- 
clusions that we came to. They gave us all of the data they had. However, I wonder sometimes about 
the actual effects of the freedom of information act, because some of the things that we wanted to 
look at clearly were there someplace but there was just no way to retrieve many of the documents 
we needed. 

The National Academy of Sciences was also very helpful. The staff they provided to the Com- 
mittee quickly acquired some knowledge of vocational education. The Academy also provided ana- 
lytical and editorial skills, and after two years of hard work by the Committee and the staff, we've 
now gone through the three layers of approval that are necessary before any National Academy re- 
port can be issued. The report should go to the printer next week. It is very much a committee 
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document: ii. lias all ihc strengths and at least a lew of the weaknesses that committee documents 
ahvays have. Already it seems to be al'tecting legislation, helping to improve administrative proce> 
dures in the research and development area and it is also affecting the plans of some professional 
organizations. We hope that it will increase the quantity and quality of vocational education R 1) 
and eventually improve the quality of service to students, both young and old. 

[low did we go about taking a iook at the vocational education research and development ol 
the last ten years? It wasn't easy. USOE had records of project titles; they had records of the 
amounts of money awarded, names of project directors and a few things of that sort. I'm not jo!;ing 
at all when I .say that most of what we wanted was in storage, and it simply seemed not to be retriev- 
able. Some of Diir best information from USOE came from interviews with the dedicated professional 
civil servants who provided a backbone and some continuity to an otherwise very rapidly changing 
structure of vocational education R&D administration. These civi) servants dug into their personal 
files, some of which were much more extensive than tlie official files. They dug into their personal 
recollections and memories to describe some of these incredible changes in administration and goals 
and structure. 

One of the most useful sources of information was provided by the synthesis papers that have 
come out of the Ohio State Center. If I have to pick a major regret about changes in vocational 
education R & D in the last few years, it has to do with the necessity for stopping the production 
of those .synthesis papers. For the period that they covered, they were just invaluable to us, and 
we continually wished for papers that covered the gap that is represented by the last few years. 

A third source of information v/as a wiiole set of commissioned state-of-the-art papers from 
researchers in this country. I was quite .-^ --.appointed, frankly, in the average quality of these state- 
of-the-art papers. Part of it was our fault Vve gave people a ^ight time line. Maybe part of it, too, 
was that we didn'l have the skills that some ol the Ohio State Center staff had acquired in learnin<^, 
how to direct the development of synthesis papers. S ;;:Tie of the papers were quite helpful to us, 
but on average, they were not as strong as they should have been. 

We investigated another source of information by holding public hearings. We invited not only 
people wlio were doing research in vocational education but also potential users of research ana 
development efforts in vocational education. We asked them a series of questions, had them pre- 
pare papers in advance, and talked with them at length about what thr y saw as some of the strengths, 
weaknesses, problems, and accomplishments. We found this source of information to be quite useful. 
Our success makes me wonder why we haven't used this public hearing technique more often when 
' are investigating needs for vocational education R&D, 

Another source of information came from staff visits to Research Coordinating Units (RCUs). 
• le RCU directors as a group were particularly helpful to us; the Southwide RCU group even con- 
ducted a special study of impact for us. 

But when you boil it right down, most of the ideas that you find in the report came from the 
Committee members, I remain convinced that a study of this type depends on having first-rate 
committee members who are v/illing to devote inordinate amounts of time (unpaid), who are dedi- 
cated to the ta.sk, and who are willing to carry it through. 

What did we learn about the past and what can we recommend for the future? Obviously you 
ought to read the entire report, and I wish I had it to hand to you today; you should be getting it 
in a month or so. I can't cover the entire report here so what I'll give you is necessarily a personal 
view of our important findings and recommendations, 
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We had recommendations and conclusions of three distinct types. The first of these had to do 
with the administration of vocational education R & D; second, the content of vocational education 
R&D; and third, the impact of vocational education R & D. I'll try covering them in that order. 



Administration of Vocational Filducation R&D 

I start off with administration because it seems to have been the easiest thing that we did. We 
really had no great difficulty in arriving at a series of rather useful recommendations for improvement 
of the administrative process in handling federal investments in vocational education R & D. As some 
examples of these recommendations/ conclusions, it was clear that in some states research and develop- 
ment were handled by different groups. Where that was the case, there was almost always no coordi- 
nation. Thus, w3 came up with an obvious recommendation that the same group within a state ought 
to operate the research and development programs.^ 

Another relatively obvious recommendation arose from the fact that in some vocational education 
research and development programs, the state decides what projects can be recommended for federal 
funding from that state. If that same state office is submitting a proposal of its own for funding, you 
have an obvious conflict of interest and indeed, in a few cases, we had state offices that refused to 
submit any proposal except their own, thus guaranteeing that they would be funded. It wasn't dif- 
ficult to recommend some changes in these procedures. 

The third administrative recommendation dealt with funding. In some of the programs of voca- 
tional education R&D, half of the funds go directly to the states and half of them are retained by 
the Commissioner. Allegedly, the Commissioner can expend these funds as he sees fit, but he has 
chosen to allocate a predetermined amount of money to projects coming from each state. A state 
which has a national center is particularly affected since the projects funded at the national center 
have been subtracted from the allocation of the Commissioner's share to that stute; or if you look 
at it the other way around, the national centers have been affected adversely because they compete 
for only a portion of the share allocated to the state in which they are situated. So in effect, it has 
been impossible for national centers to be national. Well, it wasn't difficult to recommend that the 
Commissioner's share indeed be the Commissioner's share and be allocated to the best projects, to 
the projects that had possibility for national impact, and that these funds not be allocated to a state 
regardless of the quality of the proposals coming from that state. 

Another recommendation on administration had to do with the regional administration of 
R&D. It seemed to us, and still seems to me, that regional offices are not the best places to manage 
research and development. For one thing, they've never been given enough staff to do it; but even 
more important. Congress is always seeking a multiplier effect with federal funding. I'm sure you 
are well aware that vocational education funds which go to states and to local education agencies 
result in the allocation of approximately five to ten dollars of local and state funds for every dollar 
of federal money that goes into those programs. A similar but smaller multiplier effect seems to 
exist in R & D. But there is no regional poHtical structure and no regional taxing structure, so a 
multiplier effect at the regional level is very difficult to obtain. 

We have made lots of other recommendations about the administration of vocational education 
R&D. Some of them suggested maintenance of the status quo. For example, we suggested that 
vocational education R&D ought to be retained in the operating bureau of vocational education, 
and we recommended that we keep AIM/ ARM at least until ERIC can support a comprehensive 
ctem of analysis and retrieval for vocational education. 



A greater number of our recommendations had to do with suggested changes. An obvious one 
was to increase the proportion of women and minority researchers. We suggested taking into account 
the quality of a group's previous research before awarding new support. It didn't take great brains to 
come up with a recommendation such as this. In fact, the wonder is why in the world it hadn't been 
done all along; but let me assure you that it is not done. We recommended increasing support to the 
re.search coordinating units in small states, which get so little money that they really don't have 
enough money to even open an office. We said, "If you're going to have an RCU, for goodness sakes 
give it enough money so that it can operate/' We also recommended that the RCUs be evaluated 
periodically instead of just receiving money automatically. I could go on with this list of administra- 
tive recommendations, but you can see that they really weren't all that difficult to develop and to 
.secure agreement on. 



Content of Vocational Education R&D 

The recommendations about the content of vocational education R&D were more difficult, in 
part because of our lack of data and in part because there were honest disagreements within the 
Committee about desirable directions to move in changing the content of research. But clearly a 
major problem related to content had to do with the rapidly changing federal priorities. If you take 
a look at the priority building procedure that has been used by the feds, typically they have assembled 
each year ad hoc groups which give them advice about priorities. Depending upon who assembles the 
group and who is in the group, you can be reasonably sure of coming up with just about any prede- 
termined recommendation that you want. Even more important than that fact, but clearly related 
to it, is the fact that as the administration of the program is changed, a new administrator comes in 
with a set of personal priorities and throws out the window everything that has been a priority before. 
In fact, in several places federal administrators have stated in writing that the priorities for this year 
should, if possible, be completely different from the priorities of last year. There is no way that you 
can mount a sustained and cumulative program of research if you are going to change your priorities 
every year. Sometimes a topic which was in high priority one year didn't reappear even once during 
the remainder of the ten-year period. Guidance and counseling was the only priority item which 
appeared regularly. I don't know whether this was cause or effect, but this also seems to be the only 
priority item which can be said to have developed a theoretical base. 

Because priorities very often were determined by administrators, they tended to relate to the 
need for solutions to administrative problems. And the question of whether or not a topic was re- 
searchable was seldom asked. The results of previous research had little or no bearing on whether or 
not research was to be continued. And, of course, previous research had little or no bearing on deci- 
sions as to the direction new research should take. We recommended, then, that priorities change more 
slowly; that there be an opportunity for researchers to have a voice in determining priorities; and 
that opportunities be provided for support for field-initiated proposals which did not happen to fit 
published priorities. We recommended that national centers have an opportunity to initiate research 
on topics which they saw as being important, and this in turn was based on the recommendation that 
we actually have one or more nationally supported centers for research, I really believe that in one 
sense, at any rate, we had not had national centers for research although we have had some places 
that had that title. You may disagree with this conclusion, but if you consider a national center to . 
be one which has funds for tackling problems of national importance, which determines some of its 
own priorities for tackling problems, and which is funded on an institutional rather than on a purchase- 
of-program basis, I don't think we have had even one national center for some considerable length of 
time. We recommended separate funding for career education research and development. We recom- 
mended that the Curriculum Center Network be funded adequately or that the responsibility for 
curriculum coordination be turned over to the national centers. As :^ is now, they don't have enough 
money to really do anything. 



Perhaps our most important recommendation related to the content of vocational education 
R&D was that at least 20 percent of our research development, and curriculum funds be devoted 
to research. If research is defined as a search for new knowledge, 20 percent doesn't sound like 
very much; but if you examine what has actually been done with R&D money, 20 percent would 
represent a substantial increase in investment in the search for new knowledge. 

We also suggested that research be conducted on the objectives and priorities of vocational edu- 
cation itself. Vocational education R & D, in our opinion, has tended to be passive and to concen- 
trate on improving the efficiency of existing programs; but we suggested an active role for research 
including the study of potential client groups and the identification of new goals for vocational edu- 
cation. I must say that our Committee really wanted to study vocational education instead of voca- 
cational education R&D. Repeatedly, we had to go back to our terms of reference because Com- 
mittee members said, "Look, let's study vocational education and only then come up with some 
recommendations for vocational education R & D." That course if action was temptiiig. But we 
had a hard enough time getting done what we had set out to do. 

Probably the greatest weakness in the content area of our report has to do with the conduct of 
research conducted by the states. We are reasonably sure, for example, that R&D money is being 
used for the operation of management information systems long after the research and development 
phase of those systems has passed. But data about the content of R & D state programs simply could 
not be assembled in a form that would allow us to tackle that problem. 



Assessing the Impact of Vocational Education R&D 

Our greatest problems came in assessing the impact of vocational education R&D and in making 
recommendations for increasing that impact. A high proportion of the vocational education research- 
ers with whom we talked seemed almost to resent our asking questions about impact. When we asked 
teachers and administrators about the impact of R & D, about all that they could see was the impact 
of curriculum development efforts; and ironically, that is the one area of vocational education R&D 
that has had its budget slashed most drastically. When we asked researchers about impact, they 
tended to stress the fact that a cadre of trained researchers had been developed through a fype of 
on-the-job training on research projects. We had the feeling that there were greater impacts, but we 
were unable to assess them. I feel strongly, for example, that state directors nowadays are much more 
likely to rely on data instead of intuition than they were ten years ago. But I can't prove it, and we 
simply could not find information assessing that kind of impact. 

Consequently, our recommendations for improving impact are the weakest part of our report. 
We recommended a management information system for vocational education research which would 
include the measurement of different kinds of impact. We recommended that RCUs and national 
centers be charged with translating research results for use by practitioners and with assisting them 
to become aware of and to use R&D. We recommended specific studies on the impact of vocational 
education in increasing profits, decreasing prices, improving the quality of work, and increasing the 
occupational mobility of former students. But I don't think we went far enough. Surely, the first 
step is to get researchers to accept the fact that it is reasonable to ask questions about the impact of 
research. Please don't misunderstand me. I don't mean that every project must have impact, but the 
program of research and development surely must have impact if continued support is to be expected. 
As nearly as we can tell, we have spent about 250 million dollars of federal money in the last ten 
years on vocational education research and development. At least that is the purpose for which it 
has been appropriated. And if the only discernible results are a few more active researchers and a 
few curriculum improvements, then we're not in very good shape. It was suggested to us continually 
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that impact ought, to be in terms of improved service to our clients. I surely don't believe that is the 
only kind of impact that we need to measure; but on the other hand, some of vocational education's 
impact ought to be in terms of improved service to our clients. 

Not all impact has to be visible, but some of it must be if funds for R & D are to continue. I 
would love to see, for example, a study of negative impact. In our research classes in the university, 
we talk about the importance of a review of the literature and stress that the findings of previous 
research are to be taken into account in deciding the direction in which you want to go. Now as I 
observe researchers, it doesn't work that way. They go ahead and do their research and then review 
the literature and write up that section after all the data have been collected. It would be interesting 
indeed to see if projects which have determined that **X" is not the way to go really have an impact 
on reducing the proportion of projects which use **X" in the near future. 

As I wrestled with the problem of talking to you about assessing impact of vocational education 
R & D, I couldn't help contrasting our problems with those described by Dean Kottman in Occasional 
Paper No. 10 which was published by this Center. You may recall that the Dean was asked about the 
impact of agricultural research. He had a ready answer. He came back in terms of nine bushels more 
wheat per acre from an improved variety of seed developed here. But he didn't stop there. He im- 
mediately went on to relate that to 15 million dollars more money per year for Ohio. He didn't 
seem to take offense at the question of whether agricultural research had had any impact, but more 
importantly, he listed seventeen ways in which people in Ohio are made aware of the research ;^oing 
on in agriculture at OSU. He didn't say, but I read into his presentation, that these seventeen methods 
each have two effects: they help to disseminate research knowledge, and they help people to become 
aware of the importance of the R & D effort which produced this knowledge. In other words, these 
seventeen steps increased the impact of R & D, and they helped people to understand where the 
knowledge came from. I don't think we in vocational education do a very good job of either of those 
two things. Is it so unexpected that if you ask a teacher of distributive education, **What impact has 
vocational education R&D had on you?" that person is unable to answer beyond perhaps saying 
"Well, I know of a curriculum development effort that was of some assistance to me." We simply 
have not worked at (a) teaching that teacher some research and development knowledge that would 
be useful personally, and (b) teaching that teacher that unless we have a vocational education re- 
search and development effort, such knowledge cannot be produced. We haven't done very well on 
either score, but then neither did agricultural research do very well for its first twenty-five or fifty 
years on this same score. Let us hope that it does not take us as long to get at this dual task. 

I'm a real believer in the beneficial effects of the survival-of-the-fittest on organizations as well 
as on organisms. Agricultural research learned some lessons the hard way. They had some programs 
that disappeared because they weren't adaptable, but some programs found means to increase their 
impact and to increase the awareness of the importance of agricultural R & D. I have hopes and ex- 
pectations for us as long as we recognize that questions about the impact of our work are not only 
legitimate but are crucial to our development 

I'd appreciate an opportunity to respond to questions. 
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QUESTIONS 



1. What are the obligations we have in vocalional education research for translating our results 
into terms which are understandable to teachers, and what are our obligations in teacher edu- 
cation to make new teachers receptive to the merits of research, much as is the case in science, 
engineering, etc.? 

. It s obvious that we have a responsibility to prepare and upgrade practitioners by helping them 
to understand sources of new knowledge and helping them to be able to use that new knowl- 
edge. But I want to correct one thing, if I said that the responsibility for the translation of 
research results into practitioners usable language should be in the hands of the researcher, I 
*iidn't mean to say that. I have a notion that most researchers are not very good communi- 
cators of research except possibly to other researchers. What our Committee suggested was 
that the national centers and the RCUs assume responsibility for this translation. It is the 
responsibility of the research program but not necessarily of the individual researcher. 



2. What are the societal indicators that we should examine in assessing what we should do in vo- 
cational education R&D? 

I can't give you a good, complete answer. I mentioned one measure— indications that approaches 
v/hich haven't worked in the past are less frequently used in the future because the earlier re- 
searchers had found that they weren't practical, and because later researchers knew that this 
was so. I'd love to see some measure of the extent to which research is cited. An amazing num- 
ber of publications in vocational education act as if nothing had been done before. There are no 
citations of previous research. I'd love to see some trend measures on citations which would in- 
dicate what types of projects or titles have been useful in facilitating additional research. I men- 
tioned that teachers and administrators were able to recognize that some curriculum development 
work had been done. It had been put into their hands; it had not appeared as by magic; so some- 
body must have had to do it. It makes me wonder what else among our research and develop- 
ment efforts could be made to be similarly visible. As an indication of how research is not now 
visible, we asked some people about research and development that they had done in their own 
state that had the greatest impact. It was interesting to find that View Decks were reported by 
several states as having been initiated in their state from scratch. In one sense you can say it 
really doesn't matter that people thought they had developed this counseling tool themselves 
rather than having adapted it from the work of other persons. But on the other hand, if you're 
looking at it in terms of support for R&D, it's important for somebody to know that this has 
not appeared out of thin air. If I had to suggest a project that I think would most appeal to 
congressmen, or more specifically to congressional staff, it would be a study which would look 
at answers to the question, "What kinds of impact should we be looking aC?'' After we had 
answered this question the next logical step would be to find ways of increasinf^ desirable im- 
pacts. 



10 

7 

O 

ERIC 



Would you invest in mechanisms lor defining what the impact indicators are or would you in- 
vest in means whereby impact data could be collected, processed, and reported? 

Well, if I were czar Td want to put .some money in both. Td obviously want to put a good deal 
more money in research because Tm conv'nced that it does have some impact and because as 
czar I don't have to be very accountable. Vd also be very aware of the fact that even as c/.ar, 
I don't control all the money sources so 1 would wan' ^o be able to demonstrate impact. How- 
ever, the demonstration of impact is important for another reason. Each program in vocational 
education necessarily has application to the real world. If you think of the R&D program as 
a game, the goal of the game right now is to get funding— to get my project funded. That's the 
goal. That's a pretty selfish, silly goal if you get right down to it. Now, don't misunderstand; 
Tm doing this all the time myself. I don't want to discard self preservation of the researcher 
as a goal, but surely we ought to have a larger goal of striving for new k'^owledge which is going 
to affect the bcliavior of some individual or some group. What we realiy mean when we talk 
about impact is changes in knowledge which will modify behavior. Don't ask me to put this in 
behavioral terms. I don't know how. As I said, I think the section on impact is the weakest 
part of our report; but the identification of this as a problem, the recognition of it by research- 
ers, may be a mighty important first step. 



To what extent do v/e worry about accountaLility at the possible sacrifice of the substantive 
area being researched? 

Obviously, it'd be the worst of all possible worlds in vocational education R&D if we spent 
all of our R&D money on trying to account for the expenditure of those funds. I hope I was 
clear in saying that I do not insist on, and I know of no one who does insist on, strict account- 
ability which says tha^ every research project must have measurable impact. What I'm saying 
instead is that the program which supports a variety of projects must have accountability, must 
have demonstrated impact. There's rather a difference here. We went through a period in 
which some of the feds said that every project had to have an evaluation; and in some cases, 
v/e spent more money on evaluation than we spent on the project. I don't think that's very 
sensiiiie, but it's equally nonsensical to say, ''Well, we're just going to go ahead and do research, 
and they ou^ht to fund us on faith." That's just about what we've been doing. Maybe I'm 
stating this too stronply, but I don't think so. 



Should we not invest dollars and time in orienting practitioners as to the "goods" of R & D? 

There was a very wise man who several decades ago said, "Ask not to the benevolence of the 
butcher and the baker, but instead inquire as to their self-intorest." One of the problems we 
have with talking to teachers about research and development is that if you really want to talk 
to them in terms that they understand, you talk to them in terms of, "How is this helping you? 
How is it affecting your self-interest?" The agricultural research which can say, Mr. Farmer, 
if you plant this variety instead of this variety, the char.ces are you'll get nine more bushels per 
acre" appeals to self-interest. Of course, we had good varieties that farmers weren't using until 
somebody got the message out to the farmers that their use really would be in the farmer's self- 
interest. I suggest that as we talk to practitioners about R & D, we simultaneously help them 
understand what it is we're doing for them, but more specifically, help them understand how 
this helps them and how it makes their job easier, makes their work more effective, makes their 
job better in at least one small way. Then I think they'll be willing to talk to you and support 
you. 
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Isn't there a philosophical tjuestioii as to llic amount of i!iiiTVciili(»u :]::\\ -n L-xtcrnal agency 
ran iiiipo.sv' on (.'dmational aueni ies? 

Do I inched recomnu-nd a hard-sell a[)pr()ach'* WolL yes and no. hi spite of l)layin^ the roles of 
r/ar and j.'odlVther, 1 don't want lo im[)oso an' *i on anybody. Hut on the other hand, I'm 
a «real beliwer ir* advert isinu accurately and le ' the customer make the choice. Sonvc- of 
you may recall in years [)ast whun some of us Ih/J some disagreements about the desirability of 
competition. I really believe in com[)eiition. 1 would hate to see a sinf^le curriculum product, 
for example, produced and pushed hard nationally, if we have two of them so that the customer 
has a choice, then I'm much happier because the customer will choose one of them; and that will 
su^^esl directions we ought to «o in further develo[)merit. Hut that implieb real choice. There 
is something there from which to choose, and it implies the availability of adequate, accurate 
information about that choice, ^'ou have to have bc4!i of them, and 1 don't think we've done 
a ver> ^ood job of [)rovidin^ either of them. 

Would you recommend, as a lesult of your Committee's experience, a [)lan to be [)ursucd with 
regard to solicitation of topics and sco[)es of work on which to do R & D? 

We did iiave some things to .say about RFPs, and there are some very interesting facts here. 
We learned that at least some research managers deliberately shorten the length of time that 
is available for the preparation of proposals because they have found that they get a higher 
propc^rtion of ^ood pro[)osals if there is a short time in which to prepare them. Now I think 
that's the wron^ measure. It s a nice administrative measure; if you don't want to review very 
many proposal^', you can come up with a ploy that will simultaneously decrease the number of 
proposals and increase the proportion of those proposals that are first rate. On the other hand, 
if you are looking for the best possible proposals, which I think is the administrative goal you 
ou^ht to be using, then the shortening of time doesn't make sense. However, we've found quite 
a number of cases in which the short length of time and the inadequate dissemination of infor- 
mation about priorities and RFPs was just a function of bureaucratic ine/ficiency. A decision 
was made to allocate something and it took a while to get it in the Fedora! Register and so on 
and so on. We did suggest thai there be m jch more reliance on field-initiated proposals so that 
[)eople have enough time to do a good job on a proposal in which *hey are really interested. We 
suggested more use of a process in which people submit a precii>, get that evaluated, and then the 
people who do the best job mty be invited to submit a full-blown proposal. The preparation 
Gf proposals is one of the major inefficiencies in our R & D effort— one institution costed this 
o'cjt and found that they were spending far more money and staff time in the development of 
proposals than they were gaining in receipts from those proposals. The process resulted in a 
negative gain as far as they were concerned and that really doesn't make very much sense. 

How do we resolve the problem of knowledgeable researchers being excluded from the priority 
establishment within vocational education? 

This is a ''knotty" problem. It exists in every funded R&D effort. If you involve knowledge- 
able people in the preparation of requests for proposals, (and you have to do it if the RFPs are 
\lo\ng lo be any good), you inevitably give some advantage to the people who have aided in the 
preparation of that RFP, unless you turn around and arbitrarily exclude them from competition 
under that RFP. Neither alternative is very good. We just don't have a good solution for this. 
I suppose ideally we would have our wisest people administering the program, and they could 
write capable RFPs; but I just don't believe that that is very likely to be the case. 
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You made a statement that one recommendation of the Committee was to set aside a percentage 
of dollars for exploration in new areas of knowledge in vocational education. Would you please 
comment? 

We didn't really elaborate on the types of new knowledge that we felt ought to be tackled ex- 
cept m a few cases, most of which I believe I mentioned in my presentation. We did attempt 
to assess the state of knowledge in nine different areas of vocational education research and 
suggested as a result of that investigation some things that we thought were worth further pur- 
suit. But wise as our Committee was, I don't think we were necessarily the best ones, certainly 
not the only ones, to suggest the areas of research in which new knowledge was apt to be most 
productive. We did address ourselves to a problem that is a very real one, and this is that you 
can classify research into low-risk/high-risk categories and low-payoff/high-payoff categories. 
Payoff is a guess, of course, and you're often more sure about the things that are likely to have 
low payoff than you are about the things that are likely to have high payoff. We suggested that 
the national centers in particular should address themselves to research to:;ics on which there 
was both high risk and the opportunity for high payoff and that the states address themselves 
(smce they face some political problems that we hopo will not in the future be faced by nntional 
centers), to the low-risk/high-payoff research topics if they could find them. Since there arctft 
many of those, instead, the state cc \\d concentrate on low-risk projects, but move as necessary 
toward the lower payoff end of the continuum. 

We deliberately did not refer to applied research in the document. There is a problem here in 
that NIE has been charged with responsibility for basic research in education. They have an- 
nounced that they intend to take over all vocational education R&D, applied or basic. As I 
mdicated, we think there ought to be some research that remains with the operating Bureau of 
Occupational and Adult Education, and we suggested that at least 20 percent of that effort be 
spent on the generation of new knowledge. 



What do you think is the reason(s) for researchers resisting/resenting the need to establish im- 
pact data and how can we deal with it? 

I hope I haven't overstated this problem. I didn't run into somebody who said, "I resent you 
asking that question." I implied their resentment from some of the answers I got. I think a 
good many researchers feel that, well, to put it crassly, society owes them a living. It's a little 
bit, I guess, like asking a professor the question, "What good are you?" A professor might re- 
sent being asked that, though I must confess that there are times when I find my job so enjoy- 
able that I ask myself, "Why in the world are they paying me to do something as nice as this?" 
Part of the resentment about questions having to do with impact is related to a fear that the 
wrong measures of impact will be used and that research will be destroyed by philistines who 
don't understand research, who will divert it and subvert it. But I don't have a good answer to 
your question. I'd love to see a study of just why researchers get a little bit up tight when you 
talk about impact. 



Would one solution be to fund more "action" research which could produce stronger impact 
data? 

Instead of talking about action research, we talked about demonstration and implied some 
action research accompanying the demonstration. As I said, we did not use the terms "applied" 
and "basic" at all. I'm not sure that this is a viable categorization. I'm reasonably sure that 
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one can say something about the intent of the research when it begins, but I'm not at all sure 
that one can predict in advance just what research is going to have apphcation and what research 
is going to result in the generation of basic knowledge. Similarly, we don't know which types 
of action research will result in action, but here the situation is usually worse because the resuks 
of action research are almost never communicated. I again say we're not talking about impact 
for every project. What we are suggesting is the necessity for demonstrating the impact of a 
program in order to justify its continuance. 



12. Did the Committee's study deal with preservice and in-service needs of vocational education 
R&D personnel? 

No, we really did not address that topic. We noted in passing that EPDA fellowships have gen- 
erated researchers. We noted that researchers have been trained on the job, both researchers 
who have previously had a background in vocational education and have become better research- 
ers on the job and people who have had training in social sciences who have learned on the job 
something about the problems of vocational education. But we did not directly address the 
types of training that we felt ought to be possessed by researchers or ought to be provided to 
researchers. 

13. How do we assure that investments are made in the more high-risk, rigorous, sustained research 
efforts that have long-range effects on societal noeds? 

This is an ambitious order. I'd be satisfied with some modest changes in society's activities. 
I'm reasonably sure that you are correct, that the process that we advoc^^ted will result in 
evolution rather than revolution. One possible exception may be our recommendation which 
had to do with the desirability of convening a national panel to look at vocational education 
in the same sense that the Willis Commission did and the Essex Commission did. These Com- 
missions, through the force of their arguments and their prestige, were able to bring about both 
legislative coercion and moral coercion that resulted in something pretty close to revolution in 
at least parts of the vocational education effort. Our Committee indeed, as I said, wanted to 
do that itself. It kept wanting to become another Essex Commission, and we kept having to 
drag ourselves back from that in order to attend to our terms of reference. It's very, very dif- 
ficult for any profession to reform itself, but evolution within a profession, on the other hand, 
is both feasible and desirable. It's really the latter that we were calling for from the profession 
and relying on this outside commission as a way of bringing about more major breaks with past 
practice and past goals. 

14. Did you look at financing vocational education research by agencies other than the Bureau of 
Occupational and Adult Education and the impact of those investments? 

No, we didn't. We toyed wi^.h the notion of doing this; but once you start down that road, 
there's almost no end to it. The armed forces spend an enormous amount of money on vo- 
cational education R&D designed for their own purposes but having some impact, or at least 
potential impact, on what goes on in the public sector. Something like twenty-nine different 
federal agencies spend money which can be interpreted as having implications for vocational 
education R&D. We chose, then, so far as expenditures were concerned to look only at what 
USOE had spent; we didn't even look at NIE. In terms of the results of vocational education 
research, we followed the pattern that had been set, and I think accurately, by the synthesis 
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papers that were developed here, and attempted to look at all kinds of products of research 
which fit into a particular category. Thus, we looked at the products but not at the funding 
of these other agencies. 



15. What is the impact of the Committee's report and to whom will it be disseminated? 

We made the Congressional Record with a non-flattering comment that said, "Why in the world 
didn't they do this in time to have helped us with this particular legislative cycle, and what's 
wron^ with these people that they don't take legislative needs into account?" We had one of 
the most interesting hassles that I've encountered in attempting to get clearance through the 
National Academy for the release of a preliminary version of our report to the congressional 
staff prior to the Academy review process. We got it done, and it did influence at least some 
versions of current legislation and, I think, will influence the final version that comes out this 
year. I must say that in some cases what happened was that sections of our report were used 
to reinforce the preconceived notions of congressional staff members, but then that isn't all 
bad. In other cases, we succeeded in making some suggestions that they hadn't come up with 
as yet. The most likely effects are going to be on the ways in which the vocational education 
R&D effort is administered at the federal level. We appear already to have resolved the hor- 
rendous situation in which the Commissioner's share was distributed on essentially a population 
base, by state, regardless of the quality of proposals. Many of our administrative recommenda- 
tions just make so much sense that about all that is needed is to get them said publicly, and 
they'll be accomplished. That's worthwhile. Copies of the report will be available. They'll 
be distributed widely to vocational education researchers and administrators, and they're 
printing enough copies so that most everyone who wants to get a copy will be able to get one. 
How much it will be used, I don't know. We had some skilled editors who have presented it 
in a form that is as readable as a report of this sort can be made to be. There's a highlight 
section that can be read quickly and that sort of thing. I'm hoping it will have some impact. 
I guarantee it's had some impact on me and I think on some of the other members of the 
Committee. Beyond that, I'm sorry; I just don't know. 



16. What are your perceptions of what research priorities ought to be for the next five years or 
some other projected time frame? 

I could come up with some personal suggestions. They certainly would not be Committee 
suggestions. I'd rather sit down and talk with you about this than I would to try to answer 
this important question just off the top of my head. But if I had to pick out one thing that I 
think we ought to be concentrating on, it is on the impact of the socioeconomic status of po- 
tential students and their differential success in the labor market as mediated by vocational 
education. I am convinced that socioeconomic status is real and changes people's lives in ways 
that sometimes are desirable and sometimes undesirable. I'm also convinced that we know very 
little about it as it affects vocational education. Just the simple fact that there's a negative cor- 
relation between socioeconomic status and verbal ability among post-secondary vocational and 
technical students, (that's the only population in which this negative correlation has ever been 
found) is reason enough for looking closely at SES, its effects on vocational education, and the 
ways in which vocational education affects it. Tm sure it has something to do with decisions 
about who is tracked into vocational education and on those who choose to be tracked into 
vocational education in the secondary school. Tm sure SES has something to say about the 
ways in which certain segments of society do and don't support vocational education. I could 
go on talking about my pet subjects for a long time, but I don't think I ought to do it here. 
Thank you. 
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